Planning Medical Therapy Using Partially Observable Markov Decision Processes

Diagnosis of a disease and its treatment are not separate, oneshot activities. Instead they are very often dependent and interleaved over time, mostly due to uncertainty about the underlying disease, uncertainty associated with the response of a patient to the treatment and varying cost of different treatment and diagnostic (investigative) procedures. The framework particularly suitable for modeling such a complex therapy decision process is Partially observable Markov decision process (POMDP). Unfortunately the problem of finding the optimal therapy within the standard POMDP framework is also computationally very costly. In this paper we investigate various structural extensions of the standard POMDP framework and approximation methods which allow us to simplify model construction process for larger therapy problems and to solve them faster. A therapy problem we target specifically is the management of patients with ischemic heart disease.